ent. For instance, the projection direction (w) is assumed to be

n to Σ^ିଵሺ࢛ଶെ࢛ଵሻ in the LDA algorithm. The density of each

LDA is formulated as expሼെ0.5઼^௧Σ^ିଵ઼ሽඥሺ2ߨሻ^ௗ|Σ|

⁄

, where ઼ൌ

the variables (in x) are mutually correlated, the covariance matrix

t be diagonal, i.e., the off-diagonal entries of Σ will not be zero.

ndent variables are mutually independent from each other, the

nding covariance Σ is thus degenerated to a diagonal matrix, i.e.,

agonal entries are zeros. If Σ is diagonal, the determinant of a

ce matrix (|Σ|) can be simplified by the product of the diagonal

ach of which is the variance of a variable. The decomposition of

nce to the product of variable variances due to the independence

variables is implemented by |ߑ௞| ൌ∏ߪ௞௜

ଶ. In this situation, the

yes discrimination analysis algorithm [Rennie, et al., 2003] can

useful. The definition of the Naïve bayes is defined as below,

ܠ௡|࢛௞, Σ௞ሻൌෑ

1

ට2ߨߪ௞௜

ଶ

ௗ

௜ୀଵ

exp ቆെ

ሺݔ௡௜െݑ௞௜ሻ^ଶ

2ߪ௞௜

ଶ

ቇ

(3.19)

agሺΣ௞ሻൌ൫ߪ௞ଵ

ଶ, ߪ௞ଶ

ଶ, ⋯, ߪ௞ௗ

ଶ൯ and ܠ௡ൌሺݔ௡ଵ, ݔ௡ଶ, , ݔ௡ௗሻ. It can

that the decomposition of the covariance matrix leads to the

e of the mutual correlation between the independent variables in

When the mutual correlation between variables is indeed minimal

ificant, this approach can deliver a model with a better and

performance.

R function for Naïve Bayes is naiveBayes in the package

Figure 3.8 shows the discrimination boundary of a Naïve Bayes

nstructed for the data shown in Figure 3.6. the use of this model

wo misclassified data points.

e logistic regression algorithm

stic regression algorithm [Walker and Duncan, 1967; Wolfram,

which is also called a logit regression algorithm, has been